1. Field of the Invention
The invention relates generally to the field of digital imaging. More particularly, the invention relates to performing automatic white balance on a digital image.
2. Related Art
As digital imaging becomes more prevalent, technology strives to achieve images and video with better resolution and color accuracy. One aim is to achieve images in which objects that appear white to the human eye are digitally imaged and output to likewise appear white. Although this may at first seem a straightforward task, it has proven to be quite difficult. The difficulty arises, at least in part, because a white object may appear to be one of many “non-white” colors, when viewed by an electronic device, when the object is illuminated by different kinds of light. For instance, if a white object is illuminated under a typical tungsten light bulb, found in everyday lamps, the object may appear to a digital sensor as being yellow or orange. On the other hand, the identical object may appear blue or even green under certain types of fluorescent bulbs. This problem plagues not only digital imaging but conventional, film-based imaging as well. In the realm of film, several manufacturers attempt to correct the orange or yellow cast caused by everyday household lights by using specially formulated “tungsten-balanced” film. Not being based in film, however, digital cameras must procure a different type of solution.
Presently, most consumer digital cameras employ CCD or CMOS sensor(s). Typically, these sensors include a great number of pixels, or picture elements, that register information about the light falling on them. To facilitate the collection of light, many of the sensors employ a small lens-like structure covering each pixel, which is called a microlens. These microlenses are typically made by manipulating a layer of photoresist that is placed upon the pixel plane.
The image sensors used in digital imaging are inherently “grayscale” devices, having no color to them. For this reason, the sensors typically employ a color filter array (CFA) wedged between the microlens and an active portion of the pixel structure, the pixel well. Typically, the CFA is constructed to assign a single color to each pixel. Digital camera manufacturers often choose among a variety of CFA architectures, usually based on different combinations of primary colors (red, green, blue) or complementary colors (cyan, magenta, yellow). Regardless of the particular CFA used, the overall aim is to transfer only a single color of interest, so that each pixel sees only one color wavelength. CFAs also attempt to reduce color artifacts and interference between neighboring pixels, while helping with accurate color reproductions.
One of the most popular and ubiquitous CFAs is called the Bayer Pattern, which places red, green and blue filters over the pixels, in a checkerboard pattern that has twice the number of green squares as red or blue. The theory behind the Bayer Pattern is that the human eye is more sensitive to wavelengths of light in the green region than wavelengths representing red and blue. Therefore, doubling the number of green pixels provides greater perceived luminance and more natural color representation for the human eye.
When subjected to light, an image sensor, which includes a CFA such as the Bayer Pattern, converts incident photons to electrons and is hence within the analog realm. Next, the stored electrical charges arising from the light hitting the sensor's pixels are converted to a stream of voltages via, typically, a built-in output amplifier. This stream of voltages may then be sent to an external or on-chip analog to digital converter (ADC). The ADC converts the various voltage levels to binary digital data, placing the process now within the digital realm. In the DSP, the many points of data may be assembled into an actual image, based on a set of built-in instructions. These instructions include mapping the image sensor data and identifying color and grayscale values of each pixel.
In a single sensor camera using a color filter array, one type of algorithm is called a demosaicing algorithm, which is used to derive color data per pixel. Demosaicing algorithms analyze neighboring pixel color values to determine the resulting color value for a particular pixel, thus delivering a full resolution image that appears as if each pixel's color value was derived from a combination of the red, blue, and green primary colors (if RGB colors are used). Thus, the assembled image can exhibit natural gradations and realistic color relationships.
Other types of algorithms allow the digital data to be further processed to achieve a particular color and intensity (or shade) associated with a specific pixel. Some of these algorithms tend to make pictures warmer (pinkish), while others produce cooler (bluer) results. Some set a default saturation level high, producing extremely bright, sometimes unnatural colors. Others choose a neutral, more realistic saturation, for greater subtlety and color accuracy.
Although such algorithms have shown a degree of utility in rendering fairly accurate color representations, room for significant improvement remains. In particular, conventional techniques and algorithms have not been able to automatically process data such that white objects appear truly white in digital images. Instead, the same problems mentioned above arise—objects illuminated by different types of light tend to have different castings such as orange, yellow, blue, or green.
Making this problem more pronounced is the difficulty in achieving satisfactory white-balance (a term to convey the concept of successfully rendering a white object as white using digital imaging) when faced with different illumination levels. In other words, conventional technology such as that described above has particular problems in achieving white balance when faced with successive scenes of bright and then dim light (and also scenes in which the type of light changes, for example, from tungsten-based bulbs to fluorescent bulbs).
In an attempt to address these shortcomings, some users of digital imaging equipment may turn to post image-acquisition software packages to correct for incorrect color portrayal. In particular, one may use commercially available programs to correct color balance and to ensure that a proper white balance is achieved. This may be accomplished, for example, by choosing one or more particular pixels of an image and manually assigning them a color value approximating true white. The software may then re-map the color values of the other pixels accordingly. These types of corrections can usually be done within the software by referring to, and manipulating, one or more histograms representing color mapping information or by adjusting one or more curves representing such information.
Although these techniques offer the user a great range of flexibility and control in determining the proper white balance of an image, the more post-acquisition corrections that are applied to an image, the more the overall quality of an image may degrade. For instance, heavy post-acquisition processing of an image may introduce unwanted digital artifacts. A similar phenomenon is known to film photographers, who recognize that a better print may be made from a good negative—better than a print made after applying multiple, albeit advanced, manipulations to a mediocre negative.
Based at least on the foregoing, it would be advantageous if users were provided with techniques that would produce an automatic, properly white-balanced image directly from the digital device itself. This image would, in turn, not require much post-acquisition manipulation. Hence, one could obtain better final prints or other forms of output. Additionally, it would also be advantageous if users could avoid undue post-acquisition processing since that processing can often be very time consuming, requiring specialized knowledge that is very software-dependent, and which involves the use of expensive software packages intended to run on high-end computer equipment.
Manufactures of digital equipment have typically attempted to address some, or all of the shortcomings mentioned above by using specific white balance circuitry, which can be activated while a camera images a white card. In other words, manufactures may build functionality that allows a user to point a camera at a white card or white object and effectively “tell” the camera that it is looking at something that is truly white. With that reading, the camera may use its circuitry to attempt to recognize other white objects and render them accordingly.
While this has shown a degree of success, it would be more agreeable to users to use a camera in which the white-balancing was done more automatically. Additionally, it would be desirable to provide white balancing of a digital image, converging on a close approximation of the correct color scene content, in a short period of time under varying illumination conditions.
Any problems or shortcomings enumerated in the foregoing are not intended to be exhaustive but rather are among many that tend to impair the effectiveness of previously known digital imaging and processing techniques. Other noteworthy problems may also exist; however, those presented above should be sufficient to demonstrate that apparatus and methods appearing in the art have not been altogether satisfactory and that a need exists for the techniques disclosed herein.